BACH TO THE FUTURE

*
A new generation of hyperinsturments will produce unique new sounds, play
unerringly by themselves, and take us

In the middle of a gala opera performance at the Pompidou Center in Paris, the
electronic percussion instrument that made cymbal crashes, cracks, sizzles, and
bells began producing music--on its own.

At the conductor's stool, largely hidden from the audience except for his
untamed Beethoven hair and waving hands, composer Tod Machover had been waiting
to walk onstage. He had written himself into the cast of his opera VALIS as
Mini, a semi-disabled computer-music creator.

The libretto Machover wrote for the opera explained his four-and-a-half-minute
scene: "Mini appears to be sculpting sounds, setting off musical structures
with the flick of his hand-he seems to be playing the orchestra of the
future."

So much for the script. Before Machover could make his entrance, the
percussion instrument began ringing out notes no one had written.

Oh, shit, Machover thought. The instrument's playing itself Computer
expert Joseph Chung, sitting to Machover's left, quickly turned off some
circuits. The phantom music stopped before anyone in the audience of
1,500 noticed. Chung later traced the problem to a glass globe: High
voltage inside the ball had sent signals to the instrument-a rare glitch that
underscored the potential for magic in the music of the future.

Today, at MIT's Media Lab, Machover and Chung are designing instruments
that will do what the electronic drum only seemed to do: artificially
think and act like a veteran musician. The new instruments, dubbed
hyperinstruments, are essentially collections of electronic components
orchestrated to work with a central computer or two.

Typically hyperinstruments don't play a note until they're plugged in and
attached to speakers. Then they hold a considerable edge over most
pieces in today's ensembles. Using a single hyperinstrument, Machover
can compose, conduct, perform, or make new instrument sounds, all at
the very same time.

The versatile new instruments are helping Machover and Chung take musical
risks. Already the musical innovators are creating sounds that no one has
ever heard and presenting them to potentially stuffy international
audiences. And they're teaching such established performers as cellist
Yo-Yo Ma to play the new instruments as well.

Even music critics are applauding the new notes. After a controversial
opening of VALIS in Paris in 1987, Machover is now enjoying a
crescendo of acclaim. "One of the brightest and most intelligent of new
American operas," writes a New Yorker critic. And a New York Times reviewer
predicts, "By the year 2000, VALIS will probably be seen as the first
populist, rock-influenced, computer-driven opera."

The lab in Cambridge, Massachusetts, where VALIS was spawned is one of
a few dozen facilities around the country where serious composers are
playing computer and piano keyboards, side by side. What is more, the
lab's technology is mature enough to allow listeners today to hear and
see prototypes for the smart instruments of 2010.

By then, predicts Max Mathews, hypermusic designer and professor emeritus
of music at Stanford University, "almost all music will be made
electronically, by digital circuits." The new instruments will never go out
of tune, adds Mathews, a former Bell Labs engineer and pioneer in computer
music. They'll unerringly produce the right notes in the right rhythm, "
so musicians will devote less time to these technical details and more
time to getting out precise kinds of sounds and phrasing and musical ideas."
In fact, Mathews predicts, the hyperinstrument of the future will be so
versatile that it will be capable of simulating virtually any musical
arrangement or style. Its range should include anything from whole
classical orchestras to rock groups to a single trombone; every user will be
able to program his hyperinstrument for jazz sax, rap songs, or
Beethoven's Ninth.

Within a decade, future automated orchestras will offer a menu of
interpretations to a composer or conductor, predicts John Rahn, professor
of music at the University of Washington in Seattle. Simply by talking to
a voice-sensitive computer, musicians will be able to choose the
orchestra size and sound that best matches what the composer is writing.
And if none of the options work, the music machinery of 2010 will take
orders for entirely new sounds that will exist largely in digital codes.

Machover has already given the downbeat to some of those instruments at
the Media Lab, where he is a professor and director of the Experimental
Media Facility, In a fourth-floor studio on the MIT campus, he strikes low
C on a newfangled 88-key electronic keyboard known as the Kurzweil
Midiboard, named for its inventor, When a musician plays the Midiboard,
there is at.first no sound; instead the Midiboard sends silent signals to
a Macintosh computer.

In the windowless, dimly lit room, the Mac processes the signals, feeding
them to an array of synthesizers--an electronic drum machine and other
pieces of hardware that together are responsible for most of the sizzle,
thump, brass, and plunk of a full orchestra. The silent electronic pulses
from the synthesizers race on to two mixers that control the blend from the
electronic ensemble. Slider switches on the mixers can bring up the
vibraphone, say, or dampen the drums, or crank all the music up to
fortissimo. In a final step, speakers convert the electronic output
from the mixers to rich sound that fills the room.

To start the process, Chung sits down at the Mac and types a few
commands about how the system as a whole should respond when someone
hits low C on the Midiboard. For this demonstration, low C is a trigger:
Playing the single C sets off an arpeggio, a catchy succession of about
16 notes, starting with C, that sound like the bass line for a far-out
rock song.

Machover starts hitting C's all over the Midiboard. The notes he plays
produce volleys of arpeggios, high and low, supplied by computer, from
a speaker across the room. The stream of notes speeds up the faster
he plays the keys until the tempo is almost inhuman. The computer
supplies the velocity, letting Machover concentrate on other parts of
the music; he controls loudness and softness, for instance, by altering
the pressure he puts on the keys.

The real virtuosity of the hyperinstrument appears in another
demonstration in which an amateur musician takes over the keyboard to
try some of the chords from "Suffering Song," a haunting, gentle
section from VALIS. The chords are simple enough for a sixth-grade
piano-lesson dropout to read from sheet music. Most of the chords at
the beginning of the piece have just three or four notes, not much
trickier than "Chopsticks." But when they're played on the keyboard, the
system produces a silent bolt ot lightning-fast computation and analysis.
And-like the strange ball ot Mini's performance-the instrument itself
becomes a lifelike extension of the performer.

As soon as the first chord from "Suffering Song" is struck, the
computer identifies the notes and silently searches through all of the
music Machover wrote for the opera. It's looking for the location in
the composition where the chord appears. Once it finds the place, the
computer checks out its instructions about what it's supposed to do
with the chord. In this case, the Mac has been programmed to splinter
the chord into a pattern of separate notes in a tricky rhythm, set by
Machover as composer.

The system makes it impossible to play a note that isn't in rhythm. In
fact, to an amateur, it seems to make performing a little too easy.
The notes come so fast, with so little effort, that at first players are
likely to think they're in Machover's Sorcerer's Apprentice nightmare: The
instrument's playing itself

"A lot of people say, 'Oh, yeah, this is absolute bullshit,"' Chung
acknowledges. "They argue that all we're doing is taking the
musicianship out of playing music, and making these toys so that anyone
could produce music.

"They're wrong," Chung says. "That's certainly not what we're trying to
do." Instead, Machover says, the whole point of all the technology of
the future "will be to sensitively enhance and expand the expressive
power" of individual players.

As Machover points out, the new instruments will enable human users to
concentrate on whatever aspects of musical creativity they choose. If
the composer wishes to alter the nature, or timbre, of notes-for
instance, whether they should sound like piano notes or vibraphone
notes-she can focus on just that aspect while the digital system takes
care of speed, rhythm, melody, and everything else. "I ask my computers
to do only the things I don't want to do," says Max Mathews.
"Anything that I enjoy doing or wish to express myself with, I do
myself and don't program that into the computer."

This approach, Chung notes, proved especially fruitful during work on
VALIS. "Once we took out the dimension of rhythm-once it no longer
mattered when you played things-all of a sudden your musical mind was
much more free to think of other things."

The "other things" are apparent in the CD recording of VALIS (from
Bridge Records, New York). As played by a keyboard pro, the notes from
the opening of "Suffering Song" seem to be coming from hybrid
instruments, exotic and changing combinations of marimba, guitar,
xylophone, and bell. The reason is that the computer program allowed
pianist Emma Stephenson to virtually change instruments on the spot by
pressing harder or softer on the keys. Normal pressure on the keys
produces the sound characteristic of a vibraphone. More pressure creates
the timbre of a bell, turning the keyboard into a percussion instrument for
a note or two.

Like pianist Stephenson, drummer Daniel Ciampolini gets backup from the
computer, too. The Mac keeps metronome-perfect time while he produces
different timbres by hitting rectangular rubber pads arranged to look like
oversized piano keys on a wide board. The sounds, all electronic
illusions, range from the timbre of bongos to wood blocks to strings.

Electronic illusions are flowing from another hypermusic technology
recently developed by Machover at the Media Lab as well. In this
innovation, musicians enter a kind of marriage through what the
researchers call a "double instrument."

Playing a duet, one partner produces pitches-musical frequencies or
tones-with a keyboard. These chords and melodies are not broadcast
through speakers, however; instead, they travel silently to a second
instrument-usually a percussion device like the one used by drummer
Ciampolini. Nobody hears the melody from the first device, the keyboard,
until the percussionist strikes the pads, adding rhythm and timbre. Only
then does the musical combination from the two partners go on to a
synthesizer and come out of the speakers as music.

Machover has already begun using another piece of futuristic musical
technology as well--a $20,000 metal glove that magnetically tracks the
position of every joint in the hand that wears it. Wires trail away from
the hand like exposed nerves leadIng to an IBM-compatible computer. The
IBM then sends information on the finger positions to an Apple Macintosh.
The Apple interprets the movements, controlling the sound that ultimately
emerges from synthesizers. With subtle finger waving, Machover can alter
the instruments' volume levels and frequencies and even create the illusion
of musical "panning," in which sound seems to travel around the room.

Double instruments and glove conducting can be heard on Machover's latest
CD recording, Flora, from Bridge Records. And the MIT composer says that
these hyperinstruments may lead to others, musical tools so sophisticated
they will make Mini's glass globe look like something from an old
Frankenstein movie,

While Machover today plays music with the help of a glove, for instance,
hypermusicians of the future will use the whole body, modulating
arrangements with a sort of dance.

As Machover sees it, the typical concert of 2010 may be a hybrid of visual
and aural sensations, an extension of today's choreography and laser
pyrotechnics at a rock show, but with two differences: The gyrations of
the performers will make part of the music. And most concertgoers will
see and hear it all from their own homes.

At 7:55 P.M., five minutes before the concert, you'll put on goggles
and a headset. They'll be connected through your home computer to a
telephone or cable TV line fed by cameras at a distant site, Screens
inside the goggles, one for each eye, will reveal the performers in 3-D.

They won't be tuning their instruments as today's players do before a
concert, because the computers will keep their instruments in perfect
pitch. The major action at the remote studio will be typing by a
computer programmer, making last-minute changes in the database holding
all the notes written by the composer. As you turn your head, your
hardware will sense your movement and change the scene before your eyes,
as if you were there.

When the music begins, the performers themselves will determine what
you see as well as what you hear. As virtual reality guru Jaron Lanier,
founder of VPL Research, Inc., in Redwood City, California, sees it,
saxes or horns will create scenes as well as notes. For instance, he
says, in addition to following a written musical score, the horns will
blow majestic mountains with B flats, or sandstorms in C minor right on
to the giant screen in your room. Musicians in funky artificial reality
DataSuits could be the horns, Lanier says, Their total body movements-an
extension of Machover's finger flexing-will translate into sights and
sounds.

That means that many more people, and possibly animals, will be able
to play along. Lanier says he has long dreamed of building instruments
for infants who would control a flow of notes by sucking on an
electronic nipple. Among animals, he adds, goats have the most potential.
"They relate well to flutes," he says, But how would they make music?
By chewing on an electronic cud. Lanier even predicts that some future
orchestra could include a chorus of kids goats and human babies.
"It'll probably top the charts in 2005," he says.

Ultimately sensors in home hypermusic systems will pick up brainwave
patterns and feed them to a computer/ conductor. If you choose, your
central nervous system will change the colors, shapes, and sounds of an
incoming concert to match your moods. Your anger, for example, could
tinge what you see with violent red.

In fact, even after the concert, speakers in your house will continue to
produce background music that reflects your mind's interior
landscape-uptempo riffs when you're happy, a dirge to match your blues.

Music that moves with the listener may be part of the score of
Machover's next opera, tentatively titled Can We Change Our Minds?
Working with artificial intelligence pioneer Marvin Minsky, Machover plans
to set the opera in a "malleable" concert hall so that what you hear
would depend on where you walked.

Audience actions in one part of the listening space would affect the
sound in other areas. Only in one designated area would you be able to
see and hear the music in its entirety and watch how the complicated
parts come together in a curious harmony. Only there would you get an
overall glimpse of the opera's main character: the human brain.

Like the best music of the twenty-first century, the opera will draw
audience members into a greater awareness of their own listening patterns,
Machover says. More than that, the music will be a magic mirror,
revealing deep secrets about the listener. Standing in the middle of that
future concert-in the midst of music you control-you'll know something
of what Mozart and Bach felt as they listened from their towering podiums.
From that spectacular vantage point, Machover says, "you'll find yourself
thinking about your mind in a whole new way."